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1 Introduction 

By the classical Nielsen-Schreier theorem every subgroup of a free group 
is again a free group. One of the goals of this paper is to study how one 
finitely generated free group C can be embedded into another free group 
F. Clearly, the type of embedding of a subgroup C into a group F affects 
left and right cosets, and double cosets of type CfC, f € F. A description 
of combinatorial and formal language properties of these complexes is the 
second goal of the present paper. 

Similar questions have been studied widely as auxiliary ones, for ex- 
ample, in papers [2], [12], [15] in relation to hyperbolicity of amalgamated 
products of groups, and in [3], [7], [8] in connection with the conjugacy 
problem in amalgamated products. 
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There exist a lot of papers about the characterization of embeddings 
of a subgroup C in a group F, but we shall mention only the closest to 
the subject of our paper. In [1] was shown that the property of a finitely 
generated subgroup C to be malnormal is algorithmically decidable in a 
free group F, meanwhile [I] provide an example of undecidability of this 
problem in a class of hyperbolic groups. Another related result was proved 
in [10], where was shown that a random finitely generated subgroup of a 
free group F is malnormal. 

The essential role in these studies plays the structure of a Cayley graph 
of a group and Schreier graphs for a subgroup C in F. In particular, the 
embedding of C in F can be described in terms of equations E(f, f) and 
E(f, g) (see section [3~Tj) . lemmas I3TTI 13. 4p . In this paper we lay down results 
on double cosets in a free group F with respect to the type of embedding 
of a subgroup C in F. The main results are the following theorems: 

Theorem 13. Ill Let F be a free group with a finite basis X, and C 
be a finitely generated subgroup of F with a Nielsen set of generators Y . 
Suppose / G F \ C and / is of minimal length in a double coset CfC. 
Then there exists a natural number k which can be effectively determined 
by C and Y such that 

1. if Cf = C {~\ f~ l C f = 1, then the multiplication in complex CfC is 
k— reduced, which means lx(ci) + lx(f) + lx{c2) — lx{c\fci) < 2fe, 
where C\,C2 6 C and lx(~) is the length of elements in F(X). 

2. if Cf ^ 1, then CfT is k— reduced, where T is a relative Schreier 
transversal of C in F relative to Cf. 

Theorem 13.151 Let C be a finite rank subgroup of a free group F(X). 
Then the set of all reduced words representing elements of CfC is regular 
in F(X). 

2 Preliminaries 

In this section, following [BJ[7j[IT], we give some basics on regular subsets, 
cosets, and Schreier representatives of finitely generated subgroups of a 
finite rank free group F. We shall use the following notation throughout 
the paper. 

Let F = F(X) be a free group with basis X = {xi,...,x m }. We 
identify elements of F with reduced words in the alphabet X U X~ l (i.e. 
X— reduced words), and let lx(f) denote the length of an element / G F, 
i.e. the number of letters of the reduced word / in X U X~ l . 
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Fix a subgroup C = (hi, . . . , h r ) of F generated by finitely many ele- 
ments hi, . . . , h r 6 F and an arbitrary element / € F. 

The set CfC = {ci/c2|ci,C2 S C} is called a double coset and / is a 
representative of this coset. 

2.1 Regular subsets in a free group and finite state au- 
tomata 

A non- deterministic finite state automaton A is a quintuple (S(A), X, 5, So,Fq), 
where S(A) is a finite set of states, X is a finite set called the alphabet, 5 is 
a map 5 : <S(.4,) x (XUe) t-> S(A), called the transition function, So C S(A) 
is the (non-empty) set of initial states and Fq C S(A) is the set of final 
states. Here e is assumed not to lie in X. 

An arrow in A. is a triple (si,x, S2), where si,S2 are elements of -S(^4) 
and x is an element of X Ue and is called the Za&eZ of the arrow. The source 
of the arrow is s± and its target is S2 . 

By a path of arrows in A we mean a sequence (s±,ui, S2, • • • , M n , s n +i), 
where n > and each itj is an arrow with source Sj and target Sj+i, for 
1 < i < n. We call si the source and s n+ i the target of the path of arrows. 
Let Wi be the label of Uj, i.e. a letter of X or e. Let u; be the concatenation 
wi ■ ■ ■ w n ; if n = 0, set ?/; = £. Then u; is called the label of the path. Let 
X* be a monoid of strings over X with identity e. The language L(A) over 
X assigned to a non-deterministic automaton is the set of elements of X* 
obtainable from the labels of all possible paths of arrows with source in So 
and target in Fq; in this case we say that L = L(A) is a language accepted 
by A. We say w is readable in A from s\ to S2, where s±,S2 € S^A.), if 
there is a path of arrows from s\ to S2 with the label w. 

A deterministic finite state automaton can be considered a special case 
of a non-deterministic finite state automaton, for which there following 
conditions are satisfied: there are no arrows labelled e; each state is the 
source of exactly one arrow with any given label from X; and So has exactly 
one element. 

By Kleene-Rabin-Scott theorem the regular language over X may be 
identified with the language accepted by some non-deterministic finite state 
automaton, or, equivalently, by some deterministic finite state automaton. 
If R = L(A) is a subset of a group F, then X is supposed to be a set of 
group generators for F, i.e. elements of R are meant to be reduced strings 
over IUI" 1 . 
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2.2 Graphs associated with a subgroup and Schreier transver- 
sals 

We shall identify sometimes an automaton A with a finite connected ori- 
ented labelled graph T with a distinguished vertex in natural way. Namely, 
take V = V(T) = S(A) as its vertex set; edge set E = E(T) corresponds to 
arrows, with induced labelling, and form subsets Sq,Fq C S(A) from dis- 
tinguished vertices. We shall also ascribe properties of automata to graphs. 

One can associate with a subgroup C two graphs: the subgroup graph 
r = Tq and the Schreier graph T* = T^; we refer reader to [TTJ for more 
details on this subject. 

Recall, that T is a finite connected digraph with edges labelled by el- 
ements from X and a distinguished vertex (based-point) 1 (so So = Fq = 
{1}), satisfying the following two conditions. Firstly, T is folded, i.e., there 
are no two edges in T with the same label and having the same initial or ter- 
minal vertices. Secondly, V accepts precisely the reduced words in XUX^ 1 
that belong to C. To explain the latter observe, that walking along a path 
p in r one can read a word £{p) in the alphabet X U X" 1 , the label of p, 
(reading x in passing an edge e with label x along the orientation of e, and 
reading x~ l in the opposite direction). We say that V accepts a word w if 
w = £(p) for some closed path p that starts at 1 and has no backtracking. 
Clearly, V can be identified with a deterministic finite state automata with 
1 as the unique initial and final state. 

The Schreier graph T* = of C is a connected labelled digraph with 
the set {Cu | u € F} of the right cosets of C in F as the vertex set, and 
such that there is an edge from Cu to Cv with a label x G X if and only 
if Cux = Cv. One can describe the Schreier graph V* as obtained from V 
by the following procedure. Let v 6 V and x E X such that there is no 
outgoing or incoming edge at v labelled by x. For every such v and i£l 
we attach to v a new edge e (correspondingly, either outgoing or incoming) 
labelled x with a new terminal vertex u (not in T). Then we attach to u 
the Cayley graph C(F, X) of F relative to X (identifying u with the vertex 
1 of C(F, X)), and then we fold the edge e with the corresponding edge in 
C(F,X) (that is labelled x and is incoming to u). Observe, that for every 
vertex v € T* and every reduced word w in X U X~ l there is a unique path 
r* that starts at v and has the label w. By p w we denote such a path that 
starts at 1, and by v w the end vertex of p w . The resulting graph is the 
Schreier graph of C in F. 

Notice that r = V* if and only if the subgroup C has finite index in F. 
A spanning subtree T of T with the root at the vertex 1 is called geodesic 
if for every vertex v 6 V(T) the unique path in T from 1 to v is a geodesic 
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path in V. For a given graph T one can effectively construct a geodesic 
spanning subtree T. 

From now on we fix an arbitrary spanning subtree T of T. It is easy to 
see that the tree T uniquely extends to a spanning subtree T* of T* . 

Let V(r*) be the set of vertices of P*. Since, in general, T* is infinite, we 
need to extend terminology from subsection 12.11 For a subset Y C V(r*) 
and a subgraph A of T* , we define the language accepted by A and Y as the 
set L(A, Y, 1) of the labels £{p) of paths p in A that start at 1 and end at one 
of the vertices in Y, and have no backtracking. Notice that the words £(p) 
are reduced since the graph V* is folded. Notice, that F = L(T*, V(T*), 1) 
andC = L(r,{l},l) =L(r*,{l},l). 

Sometimes we shall refer to a set of right representatives of C as the 
transversal of C. Recall, that a transversal S of C is termed Schreier if 
every initial segment of a representative from S belongs to S. 

Further, let S be a transversal of C. A representative s G S is called 
internal if the path p s ends in T, i.e., v s G V"(r). By S- in t we denote 
the set of all internal representatives in S. It follows from definition, that 

l^intl = \V(T)\. 

A representative s € S is called geodesic if it has minimal possible length 
in its coset Cs. The transversal S is geodesic if every s G 5 is geodesic. 
Clearly, if T* is a geodesic subtree of T* (and hence T is a geodesic subtree 
of T), then S is a geodesic transversal. 

3 Main results on double cosets 

In this section we study double cosets from different points of view. 
3.1 Basic definitions and properties 

Let F = F(X), C as above. Fix an element / G F \ C and consider the 
double coset CfC for this element. Clearly, the following conditions are 
equivalent: 

geCfColduckeC g = d 1 fd 2 
& 3 di, d 2 G C dig = fd 2 . 

Denote by E(g, f) the equation xg = fy over C. Due to the latter 
equivalence, an element g belongs to CfC iff the equation E(g, f) is solvable 
over C, i.e. exist c\,c 2 G C such that x = c±,y = c 2 . Lemma [3?T1 describes 
some properties of solutions of the uniform equation E(f, /). 
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We assign to / a subgroup C f C F such that C f = CC\C f , where C f 
is the set of all elements of the form f~ l C f . It turns out that Cf plays a 
special role in the whole theory of double cosets. This subgroup is one of 
the most important characteristic of embedding of C in F. 

Further, a subgroup C is called f—malnormal if Cf = 1. Recall that a 
subgroup C is said to be malnormal if it is /— malnormal for all / £ F \ C. 
If, in contrary, Cf is nontrivial then, following [T], we call / potentially 
normalizing (or pn for short). 

Lemma 3.1. Let C < F and f € F \ C . Let D(f, f) C C x C 6e i/ie se£ 
0/ aZZ solutions of equation E(f,f). In the notation above 

1. D(f,f) consists of pairs (c,c^ ) for all c € C/; 

2. z/C is f—malnormal (malnormal a fortiori), then §D(f, /) = 1; 

3. i/j € CfC, then C g is isomorphic to Cf. 

Proof. To show 1. take an arbitrary c\ € Cf, then by definition c\ € 
C* fl C . Therefore, exists an element C2 such that c\ = f~ l C2$ ■ Evidently, 
(ci , c{ ) provide a solution of E(f, f) and all such elements can be obtained 
analogously. 

Let us show 2. Suppose there exist two different solutions of E(f,f), 

f—i f-i 

say ci,C2 and d\,d,2 in C \ {1}. Then c\ = c 2 , d\ = d 2 and therefore 
(d^ciY = C2d 2 ~ 1 ■ From this it follows that d^ l c\, C2d 2 1 € Cf = 1. Thus 
ci = (Zi and C2 = CZ2 and the set D{f, f) consists of only one pair. 

To prove claim 3. suppose g = d\fd 2 for some d\, (Z2 £ C. By definition 

c 9 = C9 n c = c dl/d2 n c = (c di y d2 nc-c^n c d ? 1 =cfnc = c f . 

This concludes the proof. □ 

Remark 3.2. If an element / is pn, then every other representative in 
CfC is also potentially normalizing by lemma [37T1 

A double coset CfC is called essential if Cf ^ 1 for some representative 
/ G F (and so for all representatives of this coset). 

Lemma 3.3. The number of different essential cosets is finite. 

Proof. The union of all essential double cosets forms so called generalized 
normalizer of C in F, i.e. the set N* F {C) = {/ € F\f~ l Cf nC/1}. This 
union is known to be finite (see, for example, [7], proposition 3.5.). □ 



Lemma 3.4. Let C < F and f, g £ F \ C. If the equation E(g, f) has a 
solution (01,02) over C , then the set D(g, f) of all solutions of this equation 
consist of all pairs (cfC\,C2Cj C2 ), where c/ E Cf. In particular, if C is 
f — malnormal, then there exists a unique solution (c\,C2) with c±,C2 E C 
such that g = C\fc2 for g E CfC. 

Proof. Notice that E(g, f) is consistent by assumption and therefore exists 
a pair (ci, C2) such that c\g = fc2- If C is /—malnormal, then Cj ~ C g = 1 
and D(g, f) satisfies the derived conclusion. Denote by M the set of all 
such pairs and suppose there exists another pair of solutions, say (^1,^2). 
Then g~ 1 d^ 1 c\g = d 2 ~ 1 C2 and /cfoc^" 1 / 1 = dic^ 1 , and therefore l C2 = 
c g S C g and dic^ 1 = Cf E Cf. This implies d\ = CfC\ and di = C2C g , but 
c g = cj 19 = cj- C2 and (^1,(^2) = ( C /Ci, C2C^ C2 ) S M. The opposite inclusion 
M Q D(g, f) is straightforward. □ 

3.2 Advantages of Nielsen set of generators 

Now suppose that C supplied by a Nielsen generating set. In this subsection 
we analyze how it affects the cancellation in cosets and double cosets of C. 
We also introduce here the notion of a relative Schreier transversal of C 
with respect to its subgroup D. 

Since arbitrary set of generators for C can be carried into a Nielsen set 
(see, for example [H] or without loss of generality it can be assumed 

that C is presented by such a set Y = {h\, . . . , h r }. Let S be the geodesic 
Schreier transversal for C in F such that the Nielsen set of generators Y 
has the following properties with respect to S. 

Properties of a Nielsen set of generators: 

i. each h £ Y U Y^ 1 can be written in the form 

h = si//(/i)s^" 1 , 

where s±,S2 £ 5*i nt are written as reduced X— words, fi(h) is an ele- 
ment of X U X~ l and 

\lx{s\)-lx{si)\ < 1; 

ii. if hi 1} hi 2 are distinct elements of Y, then the letters ^{hi^ 1 and 
n{hi 2 ) ±l do not cancel on computing the X— reduced form of hf^hf^; 

iii. if hi x , hi 2 , hi 3 are elements of Y U Y~ l such that h{ 2 7^ /i^ 1 and 
hi 2 7^ hj 3 , then the letter /x(/ij 2 ) does not cancel on computing the 
X— reduced form of h^h^h^. 
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Clearly, such S always exists; and the letter fi(h) is called the central 
letter ofUFur 1 . Set M = M(Y) = [\max{l x {hx), l X {h r )}] + 1. 
The following material (up to lemma [3T7]) can be extracted from [I]. 

Lemma 3.5. J3 Lemma 2] Suppose f £ F is pn and has minimal length 
in CfC. If fc\f~ l = C2, where c±,C2 £ C,c\ ^ 1, then at least one of the 
letters of c\ appears in the reduced form of fc\f~ l , i.e., in C2- 

Lemma 3.6. JjJ Lemma 3,4] Suppose f is of minimal length in fC and 
that c = hi 1 . .. hi n (where hi x , . . . , hi n € Y U Y~ l ) is Y —reduced. 

1. Then either ^(h^) remains in fh^ . . . hi n and in this case the cancel- 
lation with f in fc is exactly that of f in fh^ ; or the central letter 
^(/ijj cancels in the product fc, and in this case hi x is of even length 
and exactly half of hi x cancels with f ; 

2. Suppose that \i{hi. ) cancels in the product fh^ . . . hi 3 . . . hi n , but ) 
does not. Then 

2.1. j < r; 

2.2. for any k = 1, . . . , j the length lx{hi k ) is even and ^{hi k ) cancels 
in product fc; 

2.3. for any k = 1, . . . ,j — 1 exactly the right half of hi k cancels 
completely in hi k hi k+1 ; 

2.4. l x {h h ) < ... < l X (hi.); 

2.5. if the right half of hi . does not cancel completely with h% j+1 , then 
hi! ■ ■ ■ hi j _ 1 sij[i(hi j ) is precisely the part of c that cancels with f 

and l x {hi x ■ ■ . h^^sijH^)) = -Ixifhj); 

2.6. if the right half of hi- does cancel with hi. +1 , then . . . hi-s is 
precisely the part of c that cancels with f for some s G S- m t and 

l x (h h ...h i]S ) < -lx(h ij+1 ). 

Lemma 3.7. ' t l, Lemma 5] Suppose that the equation E(f,f) is satisfied 
for C\,C2 € C, and f is of minimal length in CfC. Then there exists 
f € CfC of the same length as f which is a product of two pieces of 
generators from Y U Y -1 . 

Lemma 13.71 allows us to obtain an algorithm which lists all essential 
double cosets: 
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Corollary 3.8. There exists an algorithm A, which given a subgroup C of 
F and a Nielsen set of generators Y = {hi, . . . ,h r } lists all essential double 
cosets CfC of F (finite in their number by Lemma [3. 3p . Moreover, A runs 
in polynomial time in M = M(Y) = [^max{lx(hi), . . . ,lx(h r )}] + 1. 

Proof. The number of different pieces of generators in Y U Y" 1 can be 
estimated as 0(M 2 ), and therefore one can form a list of all possible prod- 
ucts /' S CfC (as in 13. T|) in time at most 0(M 4 ). Let us fix an element 
/ £ F \ C of this list. By |16t Theorem 1.6], we can construct subgroup 
graphs Yc f and Tq in time Q(N + AM) and Q(N), correspondingly, where 
N < 2Mr and Q(x) = 0(xlog*{x)). Here the function log* : N — > N is given 
by log* (a;) = A;, where k = min{m E N : log m {x) < 1}. The intersection 
of subgroups Cj and C can be constructed in time 0((N + 4M)iV), which 
is bounded above by 0(M 2 ). If now the component of Cf fl C containing 
ley x lc* is not an isolated vertex, then / is essential. □ 

Using properties of Nielsen set of generators and lemmas 13.51 13.61 we 
want to introduce a notion of the relative Schreier transversals. We shall 
use it to simplify multiplication and representation of elements in double 
cosets. 

For every pair hi,hj S Y ±l , hi ^ /ij 1 , define Ojj to be the initial 
subword of hi that does not cancel on forming the product hihj, and let bij 
be the terminal subword of hj that does not cancel on forming hihj. 

Denote also the subword of hj that does not cancel on forming 
the triple product hihjhk, where hj ^ h7 l ,hj ^ h^~ . Notice that by 
definition of a Nielsen set of generators all words a^, bij and in F(X) 
are nontrivial. Denote by £ this new alphabet {ajj} U {b^} U {rriij^} U {hi} 
obtained from all such generators, their pairs hi,hj and triples hi,hj,hk- 
We also use additional subdivision of rriijk- Namely, let rriijk = onj o fj,j o /3jk 
for some otij,(3jk € F(X) (not necessarily non-trivial) (in Section 13.31 we 
subdivide a'-s and /3's into smaller pieces: rriijk = ®iij ®2ij°fJ<j P2jk f3ijk)- 

A nontrivial S— reduced word u is called C -admissible if it has one of 
the following forms 

c = a ili2 o m ili2i3 0...0 m ik _ 2 i k _ lik o b ik _ lik or (1) 

c = a ili2 o b ili2 or c = hi. (2) 

Clearly, there is one-to-one correspondence between C-admissible words 
and nontrivial X— reduced words c € C. 

Further, let D be a nontrivial subgroup of C and suppose D is given 
by a finite Nielsen set of generators Z = {di, . . . , d m }, where d\, . . . , d m are 
Y— reduced words. Consider a subgroup graph Tp, and take a maximal 
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subtree T of Tjj such that the corresponding Schreier transversal T respect 
the choice of central letters fJ,(di). Denote by Jj n t the set of all inner rep- 
resentatives from T. Combining properties (i), (ii), (iii) of a Nielsen set of 
generators with formulae (HJ) - ([2]) above, we get 

di = h- n(di) ■ t^ 1 

= ( a «ii 2 ° m iii2h ° • • • ° m ifc-2*fe-i*fe) ° m «fe-i«fe«fe+i ° ( m i k ik+lik+2 ° ° r 

d% — Qi\%2 ^i\%2 ^ di — hi 1 , 

where ti,t 2 € T- mt ; fj l (d i ),h il £ Y U and //; ;;/ , € E. 

The system To of all C— admissible words i € T is called the relative 
Schreier transversal of C in i* 1 with respect to and T. 

Example. Consider an example of a relative Schreier transversal for 
a subgroup C < F(a,b). Let Y = {a 3 ,b 3 ,ab 2 a~ 1 ,ba 3 b~ 1 ,bab 2 a~ 1 b~ 1 } = 
{hi, h,2, h%, h,4, h$} be a Nielsen set of generators of C. The subgroup graph 
Tq is shown on figured! the edges entering maximal subtree are highlighted. 

he, 




Figure 1: Stallings automaton Tq recognizing C = 

{a 3 , b 3 , ab 2 a~ l , ba 3 b~ 1 ^a^a^b^ 1 ) 

Further, represent elements of Y U Y~ l in the form hi = su o m o 
(uniquely determined by S): 

hi = a o a o a = hg l , h>2 = b o b o b = hj l , 

/13 = ab o b o a" 1 = h$ , /14 = ba o a o ab = hg l , 

h,5 = bob 060 a~ 1 6 _1 = h^ . 

Taking all suitable products of hi, hj and hk for i,j,k = 1, . . . , 10, one 
can easily construct E. For instance, an = a 3 , 074 = b~ 2 , m.123 = b 3 , 
r«742 = a 3 , 642 = b 2 etc. The element a is pn and a subgroup C a is 
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generated by a Nielsen generating set Z = {h 2 1 /i4/i2, hi, h 2 } = {di, d 2 ,ds}. 
Then, taking a maximal subtree T as shown on figure 2(a) , obtain 

di = a 74 o m 74 2 o 6 42 , d 2 = hi and d 3 = a 22 ob 22 . 




(a) Subgroup graph Fd (b) Consolidated graph F' D 



Figure 2: Subgroup graph Fd and consolidated graph T' D for D = 
(h7, 1 h 4 h 2 , hi, hi) 

Therefore, m-742, hi and 022 are central words and 1,074,64^,622 forms 
the inner part of the relative Schreier transversal for C in F with respect 
to C a and T. This relative transversal in terms of X = {a, b} is shown 



on figure 2(b) (and we refer reader to [3] for more details on consolidated 
graphs) . 

In the sequel, we omit a maximal subtree T from the notation assuming 
it fixed for every relative Schreier transversal. 

In lemma [3^1 we showed that every representative g G CfC has a unique 
presentation in the form g = C1/C2, C\,C2 S C when Ct = 1. However, this 
is not the case for essential cosets and such a presentation is not unique in 
general. Nevertheless, the representatives of essential cosets can be uniquely 
written in a similar way: 

Lemma 3.9. Let CfC be an essential coset and T is a Schreier transver- 
sal for C in F with respect to Cf. Then every g 6 CfC has a unique 
presentation 

g = cft, aeC,teT. 

Proof. Let g = ci/di be a representative of a some essential double coset 
CfC, ci,di G C. By definition of a relative Schreier transversal there is 
an element ti 6 T such that g = ci/ii. Now suppose that g = cifti is 
another presentation for g, c 2 G C,t 2 G T. Then cifti = c 2 ft 2 implies 
c^ 1 c 2 = ftit 2 ~ 1 f~ 1 and since tit^ 1 G C we have c^ 1 c 2 G Cf and ti = Cft 2 . 
But ti,t 2 are both representatives of Cf in C and since they are in the 
same coset, ti = t 2 and hence ci = c 2 - □ 
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3.3 Cancellations in complex CfC 

In this subsection we estimate the size of cancellations in a double coset 
depending on a subgroup Cf. 

Let Y = {hi, . . . , h r } be some fixed set of Nielsen generators for C. 
Remind that M = \hmax{lx(hi), . . . , lx(h r )}] + 1 and set p being equal 
to the number of elements in the ball of radius 2M in F(X). We shall use 
this notation in what follows. The following technical lemma turns out to 
be a crucial argument in the proof of theorem 13.111 

Lemma 3.10. Let f , fi = h il f h jl ,. ..,/& = h ik f k _ 1 h jk be a sequence of 
elements in F(X) such that hi x , hj x , . . . , hi k , hj k are elements of Y U Y -1 ; 
hi k . . .hi x and hj ± . . . hj k are Y— reduced words, and lx(fi) < 2M for all 
I = 1, . . . , k. Then for all k > p there exist I, n G {1, . . . , k} and nontrivial 
ci, c n ,di,d n € C such that I ^ n, and ci ^ c n , di ^ d n , and 

Clfldl = C n fld n = C n f n d n = Clf n dl. 

Proof. Fix a number k; notice that M is at least 1 so p > 4m 2 + 1 for a 
free group F of rank m and therefore k > p > 17. In particular, k > 0. 
The number of all elements of length not greater then 2M is equal to p 
by definition, and so by Dirichlet's drawer principle there are at least two 
equal elements among k + 1 > p elements of such length, say // and f n with 
/ < n. The statement of the lemma is obvious for fi = f n = 1, so suppose 
fi,f n are non-trivial. Since 

K k ■ ■ ■ hijohj-, . . . h jk =h ik ... h il+1 fih jl+1 . . . h jk = f k , 

one can take q = hi k . . . h^ , c n = hi k . . . hi n and d\ = hj t . . . hi k , d n = 
hj n ...hi k . Clearly, ci,c n ,di,d n are non-trivial (as Y— reduced product of 
non-trivial elements ) and q ^ c n , di ^ d n . Indeed, let, for instance, 
then the product c t l c n is equal to h il+1 ...h ik . But h it+ i . . . h ik 
can not be trivial as Y— reduced word in Nielsen generators. Therefore, 
c ifidi = c n f[d n and other equalities are straightforward. □ 

Let A±, Ai, ■ ■ ■ , A n be a finite number of subsets in F(X) and consider 
the function cn : A\ ■ Ai ■ . . . ■ A n i-> N which computes the total amount of 
X— cancellations in the product a± ■ . . . ■ a n : 

cn(a 1 ,a 2 , . . . , a n ) = ^ Ix(<h) ~ lx(a\a 2 ■ ■■■■ a n )^ . 

Observe that for n = 2 the function cn(a\, a 2 ) coincides with the notion 
of Lyndon-Chiswell-Gromov product (ai,^ 1 ) in a group F with respect 
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to canonical length function lx (see |13[ [5] for details on length functions, 
and in [9] this definition was adapted to hyperbolic metric spaces). We say 
that the product A\ ■ A<i ■ . . . ■ A n is k— reduced if there is a constant k > 
such that for every aj £ Ai, i = 1, . . . , n the following holds 

m(ai, Gt2, • • • , a n ) < A;. 

For instance, two singletons j4i = {-u}, A2 = {v} such that uv = uov form 
a 0— reduced set, but we shall omit the prefix 0— below (saving a k—" only 
for k > 0). 

Theorem 3.11. Let F be a free group with a finite basis X and C be a 
finitely generated subgroup of F with a Nielsen set of generators Y . Suppose 
f £ F \ C and f is of minimal length in a double coset CfC, and let 
k = 2pM. 

1. If C is f —malnormal, then CfC is k— reduced. 

2. If CfC is essential, then CfT is k— reduced, where T is the relative 
Schreier transversal of C in F relative to Cf. 

Proof. Let g 6 CfC. If / does not cancel completely in g = cfd, then by 
lemma [3761 the length of cancellations cn(c, /) is bounded above by M (as 
well as cn(f,d) < M). Hence cn(c,f,d) = cn(c, f) + cn(f,d) < 2M and 
both (1.) and (2.) follows. If, on the other hand, / cancels completely then 
there are the two cases Cf = 1 and Cf 7^ 1 to be considered. 

1. Suppose Cf = 1 and g E CfC such that the total amount of cancel- 
lations in cfd = g is greater than k for c = h{ x . . . h{ k , d = hj 1 . . . hj n £ C. 

Let / = /' o where /' cancels completely in cf and /" cancels in 
fd. Below we list all possible forms of pieces /' and /". By lemma [3761 / ;/ 
possess the following properties: 

1.1. either {f")~ l = s, where s € Si n t and s is initial segment of s±j 1 . 
Then f'd = jhj 2 ■ ■ ■ hj n and 7" 1 is a beginning of hj^ of length 
lx(l) < lx(h n ) < 2M, and l x (f") < M; 

1.2. or (f r 1 = s lnf n. Then f'd = s^h j2 ■ ■ ■ h jn and lx(s^\) < M, 
W) < M; 

1.3. or if") 1 = a jlh o m jlhjs o m hhj4 o ■ ■ ■ o om js _ 2js _ ljs oao in 
which case f'd = f3hj s ■ ■ ■ hj n ; recall that a, f3 are elements of F{X) 
such that n r ij 3 _ 1 j s j g+1 = a o fij s o (3. Here the length lx(f") < M (see 
lemma [Ml (2.5)) and l x {P) < M; 
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1.4. or (/") 1 = a jlj2 o m jlj2ja o m j2jaji o ■■■ o om ja _ 2ja _ lja o a u where 
f'd = a 2 n ja /3h ja+1 ■■■hj n ; and ai,a 2 ,/3 are such that rrij a _ 1 j a j s+1 = 
a\a 2 o i_ij s o (3. Here the length lx(f") < M (see (2.6) of lemma [3761) 
and l x {a 2 »jj) < lx{h js ) < 2M. 

Similarly, /' and cf have one of the forms: 



2.1. either (/') = s, where s € Si n t and s is terminal segment of s 2 



Ik- 



Then cf = hi x ■ ■ ■hi k _ 1 n j and 7 1 is a beginning of hi k of length 
Ixin) < 2M, and l x (f) < M; 

2.2 or {ff 1 = Mi s 2V Then /' c = ^ii ■■■hi k _ 1 su k and Zx(sii fc ) < M, 

2.3. or (Z')- 1 =1^0 00 m ilil+lil+2 o m i;+1 ; i+2ii+3 o • • • o m ife _ 2 i fc _ life o 6 lfc _ life , 
in which case c/' = h% l - ■ • hi^a; here a, /3 are such that rn% l _ 1 i l i l+1 = 
a o fi k o /3. Here the length lx(f) < M and l x (a) < M; 

2.4. or =/32mi i i i+1 i i+2 omi i+lii+2 i i+3 o...om ifc _ 2 i fc _ 1 i fc o^ fe _ lifc , where 
c/' = hi 1 • ■ ■ hi l _ 1 a o o /3i; and f3i,f3 2 ,a are such that rrii l _ 1 i l i l+1 = 
ao fi i[0 Pi(3 2 . Here the length lx(f) < M and Ixiamfli) < 2M. 

Summarizing conditions (1.1) - (2.4) and renumbering h^, . . . , hj n for 
notational simplicity if necessary, we obtain 

g = h h --- h ik f'f"h h ■ ■ ■ h jn =h h --- hi^ iqi q 2 h j3+1 ■ ■ ■ h jn , (3) 

where 



qi 



71, see (2.1), and l x (ji) < 2M, 
s u , see (2.2-2.3), and l x (su) < M, 
, sufiiPi, see (2.4), and lx(siilMPi) < 2M, 



and 



72, see (1.1), and l x {^ 2 ) < 2M, 
q 2 = < s^j , see (1.2-1.3), and lx(s 2 j) < M, 

s a 2 HiS^ % see (1.4), and lx{pL 2 ms^j) < 2M. 

If <Zig2 = #1 92, then again cn(c,f,d) = cn(c, /) + cn(f,d) and the 
total cancellations in cfd are bounded above by 2M, in contradiction with 
assumption cn(c,f,d) > k. Therefore, q\q 2 is not reduced. Notice that 
/ ^ C implies g ^ C and therefore (71(72 ^ C. Suppose 51(72 cancels in 
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q[ o q' 2 , where q[ is an initial part of qi, q' 2 is a terminal part of qi- Without 
loss of generality one can assume that all , . . . , hj n cancels in 

cfd = h h --- h ix _^x ■ q'2hj s+1 ■ ■ ■ h jn . 
By definition of the function cn we have 

cn(c,f,d) = ±(l x (c) + l x (f)+l x (d)- l x (g) ) , (4) 

where 

k k-l 

lx(c) =^2lx(h it ) -2^2cn(h it ,h it+1 ), 
t=i t=i 

n n—1 

l x{d) = ^2lx(h jz ) -2^2cn(h jz ,h jz+1 ), 

2=1 2=1 

and l X (f) = lx(f) + lx(f"). 

Rearranging summands in @ and ([3]) we get 

cn(c, /, d) = cn(h k ■ . . . ■ h ik , /') + cn(f", h h ■ . . . ■ h ja )- 

l-l n-l 

-^2cn(h it ,h it+1 ) -^2cn(h jz ,h jz+1 )+ 

t=l z=s 
if l-l n \ 

+^[lx(q2) + lx(qi) + J2 lx ^ + Yl l x(h jz )-lx(g)\ ■ 

V t=l z=s+l J 

Further, formulae (1.1) - (2.4) imply cn(/ij ; •. . ■■hi k , /') < M, cn(f", hj 1 ■ 
■ ■■ ■ h js ) < M, and l x (qi) < 2M,l x {q2) < 2M. By definition of a Nielsen 
set of generators cn(hi, hj) > and l x (hi) < 2M for all hi, hj € Y U Y . 
Therefore, 

k < cn(c,f,d) <(l + n-s + 3)M - hx(g)- 

Moreover, the assumption that all h^, . . . , hj n cancel in © implies I — 
1 = n — s and 

9 = a'hn ob jih> ( 5 ) 
where a' iii2 is an initial segment of 6 X, ^ - 2 is a terminal segment 
of bj n _ 1 j n S S and g is not C— admissible. Hence, (11,12) 7^ (jn-i,jn) and 
since Zx(<?) > 0, we have 

0<l x (g) <4M(l + l)-4pM =► l>p-l. (6) 
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Consider the sequence 



fo 

h 
h 




fi-i 
fi 



hi 2 fi-2hj. 



'3n — s ' 



Here l x (fo) = lx(f) + lx(f") and by (1.1) - (2.4) we have l x (fo) < 2M 



and from formula (0) one can easily deduce lx(fi) < 2M. 

Further, since f\ = q[ oq' 2 , and c, d are Y — reduced words in the Nielsen 
set of generators Y (see properties (i) - (hi)), the equality ([5]) is possible 
only if all lengths lx(fi), ■ ■ ■ , Ix(fi-i) are bounded above by 2M. 

Observe that I > p — 1 due to ([6]), and fo, ■ ■ ■ , fi represent the same 
double coset. Hence by lemma [37X1 (3.) we have Cj = Cf ~ . . . ~ Cf t ~ 



Therefore, fo, . . . , fi satisfy the assumptions of lemma 13.101 and hence 
there exist elements, say fi, fj,Ci,di,Cj and dj such that q ^ Cj,di ^ dj, 
but Cifidi = Cjfidj, a contradiction to the uniqueness of the representation 
of fi in CfC (see lemma [3~4j) . 

2. Let Cf 7^ 1. Then by lemma I3T91 there is a unique presentation 
g = cfd for c 6 C and d £ T for the relative Schreier transversal T = Tc f 
(recall that we hxed a maximal subtree T in Tc f ', hence T is unique). 
Then considering elements d € T and arguing as above, obtain the equality 
9 = cifidi = Cjfidj for different indexes i,j again, a contradiction with 
lemma 13.91 □ 

Remark 3.12. Notice that for essential double cosets the complex CfC 
might not be /c— reduced for any k. Indeed, let w be a primitive element 
of an arbitrary finite rank free group F, and let C be the subgroup of F 
generated by w l for some I > 1. Then / = w is pn and does not belong to C . 
However, the total amount of cancellations in the product w kl ww~ kl S CfC 
can be arbitrary large. 

3.4 Formal language and automatic properties 

In this subsection we investigate connections between bounded cancella- 
tions and regularity of subsets of F{X). We establish the regularity of all 



C fi+i ~ C 9- 
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reduced words representing elements of a double coset CfC in a free group 
F(X) using these relations. 

Theorem 3.13. Suppose A\,A2 are two regular subsets in F . If the product 
A1A2 is k— reduced, then the set A1A2 of all reduced words in A1A2 is 
regular in F(X). 

Proof. Let Ai = (S(Ai),X,5i,So(Ai), Fo(Ai)) be deterministic automata 
accepting A^ in particular, it means Sq{Ai) = {so(Ai)} ,i = l,2. One can 
easily form a (non-deterministic) automaton A, which accepts the concate- 
nation A\A%. Namely, take the same alphabet X, S(A) = S(Ai) U S(A2), 
the new start state is so(Ai), the final states will be F (A) = Fq(A\) U 
Fq(A2), with the union of all labels and additional e— transitions from all 
states Fq(Ai) to so(-^2)- Since A1A2 is A;— reduced, the length of cancella- 
tions between the elements a\02, where a\ € A\,a,2 S A2 is bounded by k. 
Therefore, a set U of words uj E F(X) such that a\ = biouj 1 , 02 = Uj 0&2, 
and a±a2 = b\ o 6 2 , where a% € Ai, hi € F{X), is finite. Since A\ is a finite 
state automaton, for every Uj € U there is a finite set P u . = {p Uj € S(A\) ■ 
u^ 1 is readable from p Uj to /o for some /o € Fo(^4i)}; by the same argu- 
ment the set Q Uj = {q Uj € S{A2) ■ u is readable from So(^2) to q Uj } is 
finite. Add e— transitions from the states of P Uj to the states of Q Uj for 
all Uj. Clearly, A1A2 C L(^4) by construction, and moreover, A1A2 = 
L(A) (~)F(X). Thus, A1A2 is regular as the intersection of regular sets. □ 

A double-based cone with bases w±,W2 is a set of all reduced words in 
F(X), starting with w\ and ending with W2- 

Corollary 3.14. Every double-based cone C{w\,W2) in a finite rank free 
group is a regular set. 

Proof. One can prove this statement using theorem 13. 13( but we shall give 
here it's direct proof. Let w\ = %i x ---Xi s , W2 = Xj 1 ---Xj t . Then by 
definition an element / € C{w\,W2) is an X— reduced word of the form 
/ = Xj 1 • • • Xi s o y ■ ■ ■ z o Xj 1 ■ ■ ■ Xj t , with y ^ x" 1 , and z 7^ xj\ Thus, it is 
sufficient to prove the regularity of a cone with b and Xjj. Indeed, 

if C(xi s ,Xj 1 ) is regular, then so is C{w\,W2) = Xi x ■ ■ ■ xi s _ 1 o C(xi s ,Xj 1 ) o 
xj 2 ■ ■ ■ xj t as it is a concatenation of regular sets in F(X). For the notational 
simplicity we construct a deterministic automaton A recognizing a cone 
C(xi s ,Xj 1 ) in F(X) = F(a,b) and for bases equal to a and b respectively. 
An automaton A recognizing C(a,b) is shown on figure [3] (a tailed arrow 
corresponds to the initial state with an arrow, and final states are labelled 
by double circles). 
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Figure 3: The automaton A recognizing the cone C(a,b). 



If now the rank of a free group F is greater than 2, A can be easily 
modified to recognize C(a, b) in F = F(X). Namely, it is necessary to add 
the states corresponding to the rest of generators X U X~ l . From every 
state q Xi we add transitions from this state to all other including this one 
and excluding q -1, the state corresponding to inverse of generator; from 

x i 

all other states excluding q x -i we add transitions to q Xi , and the result 
follows. □ 

Theorem 3.15. Let C be a finite rank subgroup of a free group F(X). Then 
the set of all X— reduced words representing elements of CfC is regular in 
F(X). 

Proof. If Cf = 1, then all X— reduced words in CfC have a unique rep- 
resentation in terms of this complex by lemma 1374} and if Cf ^ 1, then 
these words can be represented by elements of a complex CfT for a rela- 
tive Schreier transversal T of C with respect to Cf by lemma I5~9l In both 
cases sets CfC and CfT are k— reduced in F(X) by theorem l3.111 Further, 
since C, fC and fT are regular in F(X), the result follows from theorem 
I3TT31 □ 

References 

[1] G. Baumslag, A. G. Miasnikov, V. N. Remeslennikov, Malnor- 
mality is decidable in free groups. Intern. J. of Algebra and Com- 
putation, 9 (1999), no. 6, 687 - 692. 



18 



[2] M. Bestvina, M. Feighn, A combination theorem for negatively 
curved groups. Intern. J. Differential Geom. v. 35, no. 1 (1992), 
85 - 101. 

[3] A. V. Borovik, A. G. Myasnikov and V. N. Remeslennikov, The 
Conjugacy Problem in Amalgamated Products I: Regular Ele- 
ments and Black Holes. Intern. J. of Algebra and Computation, 
17 (2007), no. 7, 1301 - 1335. 

[4] M. Bridson and D. Wise, Malnormality is undecidable in hyper- 
bolic groups. Israel J.of Math., 124 (2001), 313 - 316. 

[5] I. Chiswell, Abstract length functions in groups. Math. Proc. 
Cambridge Phil. Soc., 80 (1976), 451 - 463. 

[6] D. Epstein, J. Cannon, D. Holt, S. Levy, M. Paterson and 
W. Thurston, Word Processing in Groups. Jones and Bartlett, 
Boston, 1992. 

[7] E. Frenkel, A. G. Myasnikov and V. N. Remeslennikov, Regular 
sets and counting in free groups. Combinatorial and Geometric 
Group Theory, series "Trends in Mathematics", 2010 Birkhauser 
Verlag Basel/Switzerland, 93 - 118. 

[8] E. Frenkel, A. G. Myasnikov and V. N. Remeslennikov, Amal- 
gamated products of groups: measures of random normal forms. 
Fundamental and Applied Mathematics, v. 16, no. 8 , 2010, 189 
- 221. 

[9] M. Gromov, Hyperbolic groups. Essays in group theory, S. M. 
Gersten (ed.), MSRI 8, Springer- Verlag, Berlin, 1987, 75 - 263. 

[10] T. Jitsukawa, Malnormal subgroups of free groups. Computa- 
tional and statistical group theory (Las Vegas, NV/Hoboken, NJ, 
2001), Contemp. Math. 298, Amer. Math. Soc, Providence, RI, 
2002, 83 - 95. 

[11] I. Kapovich and A. G. Myasnikov, Stallings foldings and sub- 
groups of free groups. J.of Algebra 248, 2002, 608 - 668. 

[12] O. Kharlampovich and A. Myasnikov, Hyperbolic groups and free 
constructions. Transactions of the A. M.S., v. 350, no. 2, 1998, 
571-613. 



19 



[13] R. Lyndon, Length functions in groups. Math. Scand., 12, 1963, 
209 - 234. 

[14] R. C. Lyndon and P. Schupp, Combinatorial group theory. Ergeb- 
nisse der Mathematik und ihrer Grenzgebiete v. 89, Springer- 
Verlag, Berlin, Heidelberg, New York, 1977. 

[15] K. V. Mikhajlovskii and A. YU. Ol'shanskii, Some constructions 
relating to hyperbolic groups. London Math. Soc. Lecture Notes 
Ser., 252 (1998), 263 - 290. 

[16] N. W. M. Touikan, A Fast Algorithm for Stalling s' Folding Pro- 
cess. Intern. J. of Algebra and Computation, 16 (2006), no. 6, 
1031 - 1046. 



20 



